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ABSTRACT 

A Pennsylvania project to improve the reading skills 
c- commercial drivers and to help tnem pass the required federal 
Commercial Driver's License (CDL) written examination was evaluated 
at five pilot sites. The project used ROAD (Real Opportunities for 
Advancement and Development) to Success, a basic skills course of 
interactive computer courseware and print instructional materials 
developed for truck drivers in the Pennsylvania Department of 
Transportation (PennDOT) . Participants were 68 of 77 workers who 
failed a sample CDL test — the Quick Assessment Test (QAT) — and who 
scored below the advanced level on the vocabulary seccion 
(approximately 9th grade level) of the Tests of Adult Basic Education 
(TABE) Locator Form; 58 completed 'che 100*hour course • Personal 
interviews were conductad. A critarion-ref erenced test, TABE survey 
forms, QAT, and CDL examination were administered to participants and 
a comparison group of 10 PennDOT workers. Scores on all tests 
increased significantly for the treatment group. At the end of the 
project, 59 percent of ROAD workers passed the CDL compared with 37 
percent of the comparison group. Recommendations were for program 
dissemination, supplemental help for low-level readers, teachers with 
workplace literacy experience and computer skills, and open 
entry/open exit. (YLB) 
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Workplace Literacy Instruction and Evaluation: 
R.O.A.D. TO SUCCESS 

Commercial drivers in the United States are required to pass a federal 
Commercial Driver's License (CDL) written exam by April 1, 1992. The CDL study 
manual, on which the test is based, is written at a 6th to 8th grade reading level. This 
report summarizes the evaluation of a pilot effort to improve reading skills of 
commercial drivers who are deficient in reading (Brown, 1990). The project's goal was 
to help commercial drivers pass the required federal written examination while 
improving their literacy skills. 

R.O.A.D. to Success Program 

R.O.A.D. (Real Opportunities for Advancement and Development) to Success is 
a basic skills course of interactive computer courseware and print instructional 
materials developed for the purpose of assisting truck drivers in the Pennsylvania 
Department of Transportation (PennDOT) to develop reading skills necessary to pass 
the test. The curriculum materials demonstrate three characteristics of functional 
literacy instruction which recent research has demonstrated to be most effective in 
workforce education programs (Rockefeller Foundation, 1990; Sticht, 1987): 
integration of basic skills instruction with technical training; using technical content and 
reading iasks that are used on the job; and highlighting the learner's role as worker or 
employee during instnjction. The core curriculum is divided into two parts: a computer 
model and a classroom model. A teacher's manual accompanies each. 

The program, funded by a U. S. Department of Education Workplace Literacy 
grant, was developed through a partnership of the Pennsylvania Department of 
Education (PDE); American Federation of State, County, and Municipal Employees 
(AFSCME); and Penn State's Institute for the Study of Adult Literacy. The 
Pennsylvania Departm.ent of Transportation (PennDOT) was the recipient of services. 
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Planning and policy decisions were made by the partners. Courseware and print- 
based materials were developed at the Institute for the Study of Adult Literacy and 
tested at five sites throughout the state, including rural and urban areas. PDE served 
as fiscal agent and coordinated the delivery of services. The evaluation of course 
materials covered the general knowledge section of the CDL manual. 

Implementation of the program 

The instnjcticnal flowchart was developed by the partners in cooperation with 
PennDOT (see Figure 1). The CDL manual and a workbook prepared by PennDOT 
wero distributed to 476 PennDOT tnjck drivers in the four target counties. After a short 
interval of time, a quick assessment test (QAT), which was a sample of the CDL test, 
was adninistered to 397 of them. About 36% failed to achieve 80% of the answers 
correci whic;h is the passing grade for the actual CDL test. Of the 142 who failed the 
OAT. ICO were given the Tests of Adult Basic Education (TABE) Locator Form. Those 
who scored below the advanced level on the vocabulary section, approximately 9th 
grade, were referred to the R.O.A.D. program. Of the 1 20, 77 had scored below the 
advanced level; of that 77, 68 enrolled in the R.O.A.D.to Success program. Of the 68 
workers, 50 were white, 14 black, and 4 Spanish American; two were female. They 
had an average age of 41 , with a range of 22 to 62. They had competed an average of 
1 1 grades of schooling. As measured by the TABE Survey Form, the average reading 
level was 3.0 with a range of 1 .1 grade level to 6.3 grade level. 

Of the 68, 58 completed the 100 hour course. Classes were held on Friday 
mornings for four hours from June, 1989, until the end of January, 1990. The 
curriculum included both pnnt-based materials and computer courseware. Sixty-two 
computers were loaned by IBM corporation. Two teachers and one counselor were 
usually at each site. All of them had experience in teaching adults. 
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QAT - Quick Assessment Test 
TABE - Tests of Adult Basic Education 
CRT - Criterion Referenced Test 
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Evaluation Design 

The partners provided advice on the development of the evaluation plans and 
monitored the implementation process. They also helped In collecting data at various 
stages. The evaluation focused on the objectives of the project which related to 
developing the curriculum materials and implementing them in four counties at five 
sites. The evaluation was both formative and summative. All partners and PennDOT 
were involved in the formative evaluation. Data were continuously gathered to help 
make decisions about developing and carrying out implementation. Feedback was 
used in modifying both the process and the products. 

The major thrust of the evaluation was on measuring the changes among the 
workers in gaining basic educational skills, but attention was also given to the 
curriculum materials and the process by which they were developed and used. Both 
qualitative and quantitative data were collected before and after the project to measure 
changes which could be attributed to the program. 

Personal interviews were conducted by the external evaluator with the partners, 
the recipient of services, teachers, student workers, county managers, and the 
curriculum development staff. Tests were administered to the workers before and 
after the program to measure increase in reading skills and knowledge about content 
in the manual. Increase in basic educational skills was measured in two ways: 1) by 
a Criterion Referenced Test (CRT), which measured growth in specific reading skills 
using content from the CDL study manual; and 2) by the TABE Survey Forms. 
Increase in knowledge from the CDL manual was measured by the Quick Assessment 
Test (QAT). Wori<ers also took the CDL exam at the end of the classes. A comparison 
group of ten PennDOT workers in another county was administered the pre- and post- 
tests for QAT and CRT. They also took the CDL test. 
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Findings 

Curriculum Materials 

Teaching by computers was a key factor in the program. Workers gained 
satisfaction from using computers; computers motivated the workers to perform the 
class assignments and to remain in the program. Teachers were reported to be a vital 
and necessary component of the R.O.A.D. program. Workers were enthusiastic about 
the support and assistance provided by the teachers. The print-based materials were 
valuable because they provided structure for the lessons, and they were useful for 
class teaching and discussion, re^ ew, and homework. 

Development of the curriculum materials required a longer tin'e than originally 
planned. Teaching at the sites occurred concurently with development of the 
courseware and print-based materials. At times, the course materials were not 
available when the teachers were ready for them. Hence, teachers expressed some 
anxiety and frustration because they had limited time to cover the material included in 
the CDL manual, and many of the workers wanted to advance to new lessons. A 
similar problem could occur in any workplace literacy program where field testing of 
the materials occurs concurrently with their development. Since this was a 
developmental project, teachers and partners felt there should have been more staff 
development opportunities. Teachers did, however, demonstrate a high level of 
ability to adapt their teaching to the curriculum materials available to them. 
Changes in workers 

Scores on all tests increased significantly for the treatment group (see Table 1). 
When compared with the comparison group, the changes were not significant for the 
Criterion Referenced Test (CRT) but they were for the Quick Assessment Test (QAT) 
(see Table 2). Scores on the TABE Sun/ey Form also increased significantly for the 
treatment group; reading levels of workers increased 3.3, grades from 3.0 to 6.3. 
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Table 1 

Mean Changes for R.Q.A.D. Workers on CRT. QAT. TABE. and Reading 
Leveis('') 





N 


Pre 


Post 


Change 


T Value 




CRT 


45 


70.9 


79.8 


+8.9 


6.1 


* 


QAT 


45 


33.4 


49.7 


+16.3 


13.6 




TABE 


44 


643.2 


700.6 


+57.4 


5.6 




Reading Level 


44 


3.0 


6.3 


+3.3 


6.8 





'Significant at .001 level 

(1 )CRT had 95 items and QAT 73; scores are number of items correct 



Table 2 

T Tests for C ^hanaes Between R.Q.A.D. Workers and Comparison Group 
on CRT a nd QAT 

N Change SD df T Value 

CRT 

R.Q.A.D. 45 +8.9 9.8 52 .21 n.s. 

Comparison 9 +8.1 15.3 
QAT 

R.O.A.D. 45 +16.3 8.1 53 3.9 

Comparison 10 +5.5 6.8 

"Significant at .001 level 
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Fifty-nine percent of the R.O.A.D. workers compared with 37% of the comparison group 
passed the Commercial Driver's License exam at the end of the program, a significant 
difference. There were no significant correlations between age and educational levels 
and the CDL scores. 

Teachers observed that all workers increased their reading skills. Job 
performance of the workers, rated before and after the program on a 5-point scale, 
increased significantly as judged by the county managers. Worke.o also increased 
their work skills and basic functional skills in their family and community. Several 
workers gave examples of job improvement and new reading experiences as a result 
of the program. A'l participants were positive about the program and felt that it should 
be offered to other workers who are required to take the CDL exam. 

The wori<ers were given an attitude survey before and after the program. They 
were asked on a five point scale how they felt about various factors related to the 
program. For most questions there were no significant changes. For all questions the 
attitudes shifted towards the intended objectives of the program. Positive changes 
were significant for attitudes about beiiig a student and gaining support from family 
members for participating in the program. They became more positive about the 
R.O.A.D. program and being able to pass the CDL exam. Attitudes about using 
computers became more positive also. Benefits included passing the exam, learning 
to read better, and keeping their job. 

Recommendations 

The following recommendations were derived from the evaluation study: 

• All participants recommended that the program be disseminated to other 
potential users. 

• Low level readers should receive supplemental help (e.g., tutors, additional 
materials) with reading in a job-related context to receive maximum benefit. 

• Use of computers was strongly recommended. 



9 



• Print materials should serve as supplementary review and discussion 
materials. 

• Teachers with workplace literacy experience should be selected. 

• Teachers also need skills in working with computers. 

• Open entry and open exit from the program should be permitted. 

A Final Note 

The partnership, based on the success in the pilot sites, reapplied for funding i i 
the second round of competition for the Workplace Literacy Grants; they were awarded 
another grant to complete the curriculum, develop a workbook and tutor's manual for 
those functioning at a beginning reading level, offer training workshops to other states, 
and deliver sen/ices on a statewide basis. Currently R.O.A.D. to Success is bemg 
offered throughout the state, divided into six regions, using computers purchased by 
PDE. A teacher and paraprofessiona* are assigned t_p each region with multiple sites 
established within the regions. PennDOT moves the computers in its vans every two 
months within each region to provide statewide coverage. Since currently about 95 
percent of PennDOTs wori<force has passed the CDL exam, most of the learners in 
the current project are school bus drivers, workers from other state agencies and 
municipalities, and private industry employees. Similar evaluation procedures are 
underway in the current project and will be available at the conclusion after 
September 30, 1991. 

While pafMership efforts are never easy, joint efforts do result in service delivery 
that is far superior to what any one organization could do alone. Union participation is 
deemed as essential to recruitment and morale among the learners. This model 
makes great sense in upgrading the literacy skills cf a workforce that had been away 
from school and literacy activities for many years. Embedding liters . iHjction in 
job-related instnjction results in better retention of literacy skills because the content is 
relevant to the learners; instruction can build upon already existing background 
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knowledge which is important in reading comprehension. Computers are essential in 
workforce education programs because they offer a face-saving way to learn literacy 
skills as well as prepare for the modern workplace. 
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